From technological innovation to industrial application

More and more virtual digital people are on duty

With the development of artificial intelligence, virtual reality and other technologies, digital virtual human has frequently appeared in the public’s view. As a virtual character that exists in the digital space in digital form and has the appearance, behavior and characteristics of personification or real person, how was the digital human developed? What are the applications?

Enter People’s Daily Life

“Hello everyone, Gu Xiaoyu is very polite!” Recently, at the opening ceremony of Deshou Palace Relics Museum of the Southern Song Dynasty, a digital virtual human in ancient costume acted as the host, attracting the attention of audience. Look carefully, this digital human wearing fluttering clothes with good looking and simple and elegant temperament, which shows flexible and lovely. It is the promoter of Song charming culture launched by Zhejiang Satellite TV, a subsidiary of Zhejiang Media Group.

“Our launch of Gu Xiaoyu is not only close to traditional culture, but also to young groups.” The head of Gu Xiaoyu Studio of Zhejiang Satellite TV said in an interview with our newspaper that Gu Xiaoyu is the first digital China-Chic product developed by Zhejiang Satellite TV. The product integrates Song rhyme elements with new media, new technologies and new applications, and innovatively creates the image of Gu Xiaoyu. “We hope to make the majority of audiences, especially young people, like Song charming culture with lovable characters and touchable digital means.”

Gu Xiaoyu has appeared on various occasions - in TV programs, it sang Song Ci and introduced Cuju culture of Song Dynasty. During the Wuzhen Summit of World Internet Conference, it cooperated with guests online via H5. In the Deshou Palace Relics Museum of the Southern Song Dynasty, it became a guide and responded to the questions of tourists. Recently, Gu Xiaoyu participated in the “One Day of Southern Song People” foreign cultural exchange cloud exhibition, which was also launched in China’s cultural centers in New Zealand, Germany, the Netherlands, Pakistan and other countries.

Similar to Gu Xiaoyu, more and more digital human are on duty in the fields of cultural tourism, finance, government affairs and other services, and enter people’s daily life. For example, the virtual digital Human Tianyu with the style of Airs of the States, based on Flying Apsaras in Dunhuang, has more than 5 million fans across the network. Xiaoxin, a new employee of CITIC Financial Holdings, who appeared at the 2022 World AI Conference, can serve as the exclusive wealth planner for customers. Check out the portal website of Xiantao Municipal People’s Government of Hubei Province, and the digital human Taotao in red and white dress broadcasts the video news and policy interpretation with a smile... The Digital Human Industry Development Trend Report (2023) pointed out that digital human are moving from technological innovation to industrial application with the accelerated development of interactive intelligence technology.

Indispensable and Interesting Connotations

How does such a lifelike digital human come out? Taking Gu Xiaoyu as an example, digital human should first be like people. According to the character concept design, Gu Xiaoyu came from Lin’an in the Southern Song Dynasty and was born in Grain Rain solar term. It has the beautiful meaning of “The rain begets all kinds of grain and everything is thriving”. Zhejiang Satellite TV specially cooperates with Song Charming Culture Research and Inheritance Center to invite relevant Song Charming culture experts to form an advisory group to provide professional guidance for Gu Xiaoyu’s character modeling, image design, and content presentation. When it appeared in a singing program, Gu Xiaoy’s stage style was a set of modern and improved Song Dynasty Hanfu, with the upper part of which was made of improved sleeved over-dress and Song Dynasty layering corset. And the cuff shape was inspired by the lotus leaves of the West Lake. The lower part of which was made of ink printed long skirt. The makeup reference was the pearl makeup of ladies of the Southern Song Dynasty.

As a technical supporter, Tencent team has created a high-precision realistic personification with more than 2 million faces to meet the diversified needs of large-scale stage program recording and intelligent multimodal interaction. Liu Mingliang, the relevant person in charge of Tencent, told the reporter that Gu Xiaoyu’s twist braid was composed of 300,000 hairs, and there were five layers of clothes made of different materials. In order to achieve the effect of both form and spirit, the team carried out real-time interactive simulation based on the game engine, making the characters’ performance under different lights and actions very real.

In addition to good looking, digital human should have interesting connotation. For example, Gu Xiaoyu successfully mastered the knowledge of Song charming encyclopedia with the help of 400G open data and 3 billion parameters of large-scale language model training before becoming a guide. In order to enable Gu Xiaoyu to have a real-time voice conversation with tourists, the technical team used voice recognition, natural language understanding, voice synthesis and other technologies to make the digital guide understand, answer and understand more.

Gu Xiaoyu also adopted the voice-driven mouth shape technology. By adapting various types of audio input such as voice, singing and mood words, and learning a wide range of emotional voice data, highly expressive voice with natural rhythm, rich emotional style and personality was generated. In addition, with flexible and accurate facial expression and mouth shape, the voice of digital human can achieve the effect comparable to that of real people.

Strong Industrial Growth Momentum

At present, digital human has been widely used in enterprise services, film and television animation, cultural media and other fields. According to The In-Depth Research And Investment Value Analysis Report of China’s Virtual Human Industry from 2022 to 2023, under the background of increasing entertainment demand and continuous iteration of artificial intelligence and other technologies, China’s virtual human industry has developed rapidly, and the upsurge of metaverse has accelerated the upgrading of virtual digital human industry. According to the data, the size of China’s virtual human-driven industrial market and core market was 107.49 billion Chinese yuan and 6.22 billion Chinese yuan respectively in 2021, and is expected to reach 640.27 billion Chinese yuan and 48.06 billion Chinese yuan respectively in 2025, showing a strong growth momentum.

The reporter learned that the production process of virtual digital human generally includes model binding, motion capture, animation solution, real-time rendering and other steps, which need to use a large number of high-tech cutting-edge technologies such as reality enhancement, in-depth learning, virtual production and so on. The relatively simple animation image digital human can be completed in one month. However, it takes 3 to 6 months or even longer to make a more realistic 3D digital human from character design to rendering. The more exquisite and lifelike digital human are, the more time and labor costs are spent.

Liu Mingliang said that technological iterations such as computing power improvement and engine upgrading are promoting the continuous improvement of digital human production efficiency and intelligence level. On the one hand, the production of digital human is cheaper and more efficient. On the other hand, digital human are becoming more and more interactive, and the attributes of “human” such as services and companionship are stronger.

Industry insiders believe that with the further development of metaverse and relevant technologies, digital human will enter a period of large-scale application and accelerate the transformation into real productivity, from focusing on animation and rendering of identity-based digital human to service-oriented digital human integrating language understanding, expression, learning and interaction capabilities, from online services to on-site experience.